Omni-stereo imaging research may involve the capture and display of stereoscopic (stereo) three-dimensional imagery for substantially all of an environment (omni), including one or more spherical images. Many techniques have been developed for capturing omni-directional monoscopic imagery of an environment using wide-angle lenses, mirrors, and various image mosaicing techniques. Similarly, many techniques have been developed for capturing stereoscopic imagery. There are even some techniques that can be used to capture stereoscopic omni-directional (omni-stereo) imagery.
Early attempts made use of two monoscopic omni-directional cameras vertically displaced along a common axis. By comparing the imagery from both cameras, depth information could be extracted from the surrounding scene. However, human eyes are horizontally displaced, rather than vertically, so the omni-stereo imagery produced by the vertical camera arrangement is inappropriate for human stereo perception.
Some techniques rely on specially constructed spiral mirrors and/or lenses. While these devices theoretically are capable of capturing omni-stereo imagery in real time, they are cylindrical, rather than spherical, in nature. Thus, they may capture 360° of imagery in the horizontal direction, but are more limited in the vertical direction, and usually unable to capture more than 90° of vertical imagery. While some of these theoretical formulations have been extrapolated into a spherical context, the resulting spherical omnivergent images are designed for automated stereo reconstruction operations, rather than human stereoscopic viewing.
Another approach, using a center-strip omnivergent sensor, can be applied more directly to human stereoscopic viewing. This sensor captures a succession of circular imaging sweeps and merges them into a unified panoramic image. Thus, a camera may be placed at successive positions around a circle, and at each position, the camera can be rotated 360° about its central axis (coinciding with a radius of the circle), so as to capture a full circle of image data. This process results in a unified panoramic image containing both forward and backward tangent rays. Decomposing the image into separate forward and backward tangent ray images permits stereoscopic viewing when one image is shown to each eye of the viewer. However, this approach also fails to capture some of the image data.
Spherical imagery is usually displayed to a human viewer by mapping images onto a spherical surface that surrounds the viewer. The viewer can then change the viewing direction interactively to explore the environment. However, when images provided by a center-strip omnivergent sensor are viewed in this manner, it becomes apparent that the tangential camera path results in a failure to capture the areas of the surrounding environment corresponding to the top and bottom apexes of the spherical field of view—that is, some areas near the apexes of the sphere are simply missing. Thus, omniscopic and stereoscopic viewing of such imagery is flawed at the apexes (e.g., above the viewer's head, and at the viewer's feet). The use of toroidal topology for panoramic imagery also fails to solve such apex viewing flaws.